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DETAILED ACTION 

Response to Amendment 
1. The Applicants' amendment, filed 7 August 2009, has been received, entered into 
the record, and considered. 



2. As a result of the amendment, claims 4 and 7 have been amended. Claim 2 has 
been previously canceled. Claims 1 and 3-12 remain pending in the application. 



Priority 

3. The Applicants' claim to foreign priority as a 371 application of PCT/IB/03/03660, 
filed 18 August 2003, which depends for priority upon European Patent Application 
EP-02078955.8, filed 24 September 2002, is acknowledged. 

The priority documents have been received and entered into the record. 



Claim Objections 

4. In view of the Applicants' amendment to claims 4 and 7, the pending claim 
objections are withdrawn. 
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Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 



6. The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 
USPQ 459 (1966), that are applied for establishing a background for determining 
obviousness under 35 U.S.C. 103(a) are summarized as follows: 

1. Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating 
obviousness or nonobviousness. 



7. This application currently names joint inventors. In considering patentability of 
the claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of the 
various claims was commonly owned at the time any inventions covered therein were 
made absent any evidence to the contrary. Applicant is advised of the obligation under 
37 CFR 1.56 to point out the inventor and invention dates of each claim that was not 
commonly owned at the time a later invention was made in order for the examiner to 
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consider the applicability of 35 U.S.C. 103(c) and potential 35 U.S.C. 102(e), (f) or (g) 
prior art under 35 U.S.C. 103(a). 



8. Claims 1, 3 and 7-12 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Billmaier (U.S. Patent 6,630,963) in view of Errico et al. (U.S. Patent 7,055,168). 

9. Regarding claim 1, Billmaier teaches a system for operating with different types 
of media content substantially as claimed, the system being arranged to enable a user to 
use a first content of a first type (see disclosure that a user watches television content, 
col. 1, lines 15-36 et seq.), characterized in that the system comprises: 

a) identifying means for identifying that the user concurrently uses a second 

content of a second type when using said first content of the first type (see 
disclosure that a user selects a secondary audio program to replace the 
primary audio program associated with a television transmission, col. 9, 
lines 1-10 et seq.), said second content being unrelated with the first content 
(see disclosure that the secondary audio content is transmitted via the 
Internet, col. 2, lines 44-51 et seq.; the examiner notes that Applicants' 
specification discloses that the term 'unrelated' should be interpreted as 
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meaning that the system has no data indicating the relation between the 
contents, page 3, lines 21-24; although the television and secondary audio 
content might both be broadcasting the same event, they are independently 
produced and broadcast, and there is no reference in either data stream to 
the other, which means that for the purposes of the claimed invention, the 
video and audio broadcasts are unrelated; the fact that the buffering period 
of the video stream must be manually calibrated in order to match the 
audio stream provides further evidence that the streams are unrelated); 

b) associating means for associating said second content with the first content 

(see disclosure that the user selects a desired secondary audio program to 
replace the primary audio program associated with the television 
transmission, col. 9, lines 1-10 et seq.); and 

c) storage means arranged to store meta-data comprising information pertaining 

to said associated first and second content (see disclosure that the storage 
device may store, inter alia, electronic programming guide (EPG) data, col. 
5, lines 10-15 et seq.; see also disclosure of synchronization data associated 
with media content, col. 7, lines 10-43) 
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Billmaier does not explicitly teach a system for operating with different types of 
media content wherein the association between said first content and said second 
content is made on the basis of historical usage of the user. 

Errico et al., however, teaches the monitoring and recording of a user's viewing 
and listening habits, and the rendering of desirable audio, image and/or video 
information in accordance with said user's historical viewing and listening habits (see 
col. 4, lines 25-63; see also col. 11, lines 34-45; see also col. 12, lines 25-32; see also 
disclosure that the management of individual programs and other data includes 
relationships between any one of the user, the audio, and the images in relation to one 
or more of a program description scheme and a user description scheme, col. 7, lines 6- 
9; see also disclosure that appliances that can be personalized may access content from 
different sources, or access multiple or different types of single media such as video, 
music, etc., col. 11, line 66 through col. 12, line 3 et seq.). 

It would have been obvious to one of ordinary sill in the art at the time of the 
invention to provide personalization of a user's multimedia viewing based upon their 
past recorded behavior, since this would allow a system to render multimedia to a user 
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in a manner consistent with that user's preferences, without the need for the user to 
manually customize the rendering each time a user accesses multimedia. 



10. Regarding claim 11, Billmaier teaches a method of operating with different types 
of media content substantially as claimed, the method comprising the step of 
identifying a user's usage of a first content of a first type (see disclosure that a user 
watches television content, col. 1, lines 15-36 et seq.), characterized in that the method 
further comprises a step of identifying that the user concurrently uses a second content 
of a second type when using said first content of the first type (see disclosure that a user 
selects a secondary audio program to replace the primary audio program associated 
with a television transmission, col. 9, lines 1-10 et seq.), said second content being 
unrelated with the first content (see disclosure that the secondary audio content is 
transmitted via the Internet, col. 2, lines 44-51 et seq.; the examiner notes that 
Applicants' specification discloses that the term 'unrelated' should be interpreted as 
meaning that the system has no data indicating the relation between the contents, page 
3, lines 21-24; although the television and secondary audio content might both be 
broadcasting the same event, they are independently produced and broadcast, and 
there is no reference in either data stream to the other, which means that for the 
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purposes of the claimed invention, the video and audio broadcasts are unrelated; the 
fact that the buffering period of the video stream must be manually calibrated in order 
to match the audio stream provides further evidence that the streams are unrelated), 
and a step of associating said second content with the first content (see disclosure that 
the user selects a desired secondary audio program to replace the primary audio 
program associated with the television transmission, col. 9, lines 1-10 et seq.). 

Billmaier does not explicitly teach a method of operating with different types of 
media content wherein the association between said first content and said second 
content is made on the basis of historical usage of the user. 

Errico et al., however, teaches the monitoring and recording of a user's viewing 
and listening habits, and the rendering of desirable audio, image and/or video 
information in accordance with said user's historical viewing and listening habits (see 
col. 4, lines 25-63; see also col. 11, lines 34-45; see also col. 12, lines 25-32; see also 
disclosure that the management of individual programs and other data includes 
relationships between any one of the user, the audio, and the images in relation to one 
or more of a program description scheme and a user description scheme, col. 7, lines 6- 
9; see also disclosure that appliances that can be personalized may access content from 
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different sources, or access multiple or different types of single media such as video, 
music, etc., col. 11, line 66 through col. 12, line 3 et seq.). 

It would have been obvious to one of ordinary sill in the art at the time of the 
invention to provide personalization of a user's multimedia viewing based upon their 
past recorded behavior, since this would allow a system to render multimedia to a user 
in a manner consistent with that user's preferences, without the need for the user to 
manually customize the rendering each time a user accesses multimedia. 



11. Regarding claim 12, Billmaier teaches a computer-readable medium containing a 
computer program product substantially as claimed, enabling a programmable device, 
when executing said computer program product, to: 

a) identify a user's usage of a first content of a first type (see disclosure that a 

user watches television content, col. 1, lines 15-36 et seq.); 

b) identifying that the user concurrently uses a second content of a second type 

when using said first content of the first type (see disclosure that a user 
selects a secondary audio program to replace the primary audio program 
associated with a television transmission, col. 9, lines 1-10 et seq.), said 
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second content being unrelated with the first content (see disclosure that 
the secondary audio content is transmitted via the Internet, col. 2, lines 44- 
51 et seq.; the examiner notes that Applicants' specification discloses that 
the term 'unrelated' should be interpreted as meaning that the system has 
no data indicating the relation between the contents, page 3, lines 21-24; 
although the television and secondary audio content might both be 
broadcasting the same event, they are independently produced and 
broadcast, and there is no reference in either data stream to the other, 
which means that for the purposes of the claimed invention, the video and 
audio broadcasts are unrelated; the fact that the buffering period of the 
video stream must be manually calibrated in order to match the audio 
stream provides further evidence that the streams are unrelated); and 
c) associating said second content with the first content (see disclosure that the 
user selects a desired secondary audio program to replace the primary 
audio program associated with the television transmission, col. 9, lines 1-10 
et seq.). 



Billmaier does not explicitly teach a computer-readable medium containing a 
computer program product for operating with different types of media content wherein 
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the association between said first content and said second content is made on the basis 
of historical usage of the user. 

Errico et al., however, teaches the monitoring and recording of a user's viewing 
and listening habits, and the rendering of desirable audio, image and/or video 
information in accordance with said user's historical viewing and listening habits (see 
col. 4, lines 25-63; see also col. 11, lines 34-45; see also col. 12, lines 25-32; see also 
disclosure that the management of individual programs and other data includes 
relationships between any one of the user, the audio, and the images in relation to one 
or more of a program description scheme and a user description scheme, col. 7, lines 6- 
9; see also disclosure that appliances that can be personalized may access content from 
different sources, or access multiple or different types of single media such as video, 
music, etc., col. 11, line 66 through col. 12, line 3 et seq.). 

It would have been obvious to one of ordinary sill in the art at the time of the 
invention to provide personalization of a user's multimedia viewing based upon their 
past recorded behavior, since this would allow a system to render multimedia to a user 
in a manner consistent with that user's preferences, without the need for the user to 
manually customize the rendering each time a user accesses multimedia. 
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12. Regarding claim 3, Billmaier additionally teaches a system further comprising 
selection means arranged to select the content (see disclosure that the user selects a 
desired secondary audio program to replace the primary audio program associated 
with the television transmission, col. 9, lines 1-10 et seq.). 

13. Regarding claim 7, Billmaier additionally teaches a system wherein said 
selection means are further arranged to user-operably modify said meta-data (see 
disclosure that the buffering period for a given media content is user-adjustable, col. 7, 
lines 26-27). 

14. Regarding claim 8, Billmaier additionally teaches a system wherein said 
identifying means is arranged to identify a user's usage of a third content of a second or 
other type, said usage being concurrent to said user's usage of the first content, and said 
third content being unrelated with the first content, and wherein said associating means 
is arranged to associate said third and first content (see disclosure of the association of 
additional content or a variety of different types, col. 9, lines 29-50), the system further 
comprising rating means arranged to rate said association of the first content with the 
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second content and/or with the third content (see disclosure of the explicit selection of 
secondary audio content, thus rating the association positively, col. 8, lines 26-30). 

15. Regarding claim 9, Billmaier additionally teaches a system comprising a 
plurality of devices, each device including output means arranged to output at least one 
type of the media content, and/or input means arranged to obtain at least one type of 
the media content (see disclosure of a system comprising a plurality of set top boxes, 
television, personal computer, advanced television set, or another type of client 
terminal, col. 3, lines 18-25, as well as a head-end centrally-located facility where 
television programs are received from local cable television, satellite downlink or other 
sources and routing video streams and other data to and from the various set top box 
devices serviced thereby, col. 3, lines 48-67). 

16. Regarding claim 10, Billmaier additionally teaches a system wherein said first 
and second content correspond to video and audio content (see disclosure that a 
secondary audio program is selected to replace the primary audio program associated 
with the television [video] transmission, col. 9, lines 1-5). 
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17. Claims 1, 3, 4 and 6-12 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Bahn (U.S. Patent 7,162,728) in view of Errico et al. (U.S. Patent 7,055,168). 



18. Regarding claim 1, Bahn teaches a system for operating with different types of 
media content substantially as claimed, the system being arranged to enable a user to 
use a first content of a first type (see disclosure that the system allows a user to 
customize audio content on interactive television, col. 2, lines 8-10 et seq.), characterized 
in that the system comprises: 

a) identifying means for identifying that the user concurrently uses a second 

content of a second type when using said first content of the first type (see 
disclosure that the system allows a user to customize audio content on 
interactive television, col. 2, lines 8-10 et seq.), said second content being 
unrelated with the first content (see disclosure that, for example, the user 
can elect to listen to jazz style music while viewing content on a shopping 
channel accessed over interactive television, col. 2, lines 33-35); 

b) associating means for associating said second content with the first content 

(see disclosure that once the user's selections have been made, the 
selections can be stored as user preferences in an audio library collection 
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and then applied during subsequent viewing of the shopping channel, col. 
6, lines 40-43 et seq.); and 
c) storage means arranged to store meta-data comprising information pertaining 
to said associated first and second content (see disclosure that the storage 
device may store, inter alia, a variety of audio content which can be 
arranged by style of music, individual song, album, artist, or by other 
classifications, col. 6, lines 1-7; see also disclosure that the user may upload 
audio and/or video content, col. 7, lines 15-51). 

Bahn does not explicitly teach a system for operating with different types of 
media content wherein the association between said first content and said second 
content is made on the basis of historical usage of the user. 

Errico et al., however, teaches the monitoring and recording of a user's viewing 
and listening habits, and the rendering of desirable audio, image and/or video 
information in accordance with said user's historical viewing and listening habits (see 
col. 4, lines 25-63; see also col. 11, lines 34-45; see also col. 12, lines 25-32; see also 
disclosure that the management of individual programs and other data includes 
relationships between any one of the user, the audio, and the images in relation to one 
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or more of a program description scheme and a user description scheme, col. 7, lines 6- 
9; see also disclosure that appliances that can be personalized may access content from 
different sources, or access multiple or different types of single media such as video, 
music, etc., col. 11, line 66 through col. 12, line 3 et seq.). 

It would have been obvious to one of ordinary sill in the art at the time of the 
invention to provide personalization of a user's multimedia viewing based upon their 
past recorded behavior, since this would allow a system to render multimedia to a user 
in a manner consistent with that user's preferences, without the need for the user to 
manually customize the rendering each time a user accesses multimedia. 

19. Regarding claim 11, Bahn teaches a method of operating with different types of 
media content substantially as claimed, the method comprising the step of identifying a 
user's usage of a first content of a first type (see disclosure that the system allows a user 
to customize audio content on interactive television, col. 2, lines 8-10 et seq.), 
characterized in that the method further comprises a step of identifying that the user 
concurrently uses a second content of a second type when using said first content of the 
first type (see disclosure that the system allows a user to customize audio content on 
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interactive television, col. 2, lines 8-10 et seq.), said second content being unrelated with 
the first content (see disclosure that, for example, the user can elect to listen to jazz style 
music while viewing content on a shopping channel accessed over interactive 
television, col. 2, lines 33-35), and a step of associating said second content with the first 
content (see disclosure that once the user's selections have been made, the selections can 
be stored as user preferences in an audio library collection and then applied during 
subsequent viewing of the shopping channel, col. 6, lines 40-43 et seq.). 

Bahn does not explicitly teach a method of operating with different types of 
media content wherein the association between said first content and said second 
content is made on the basis of historical usage of the user. 

Errico et al., however, teaches the monitoring and recording of a user's viewing 
and listening habits, and the rendering of desirable audio, image and/or video 
information in accordance with said user's historical viewing and listening habits (see 
col. 4, lines 25-63; see also col. 11, lines 34-45; see also col. 12, lines 25-32; see also 
disclosure that the management of individual programs and other data includes 
relationships between any one of the user, the audio, and the images in relation to one 
or more of a program description scheme and a user description scheme, col. 7, lines 6- 
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9; see also disclosure that appliances that can be personalized may access content from 
different sources, or access multiple or different types of single media such as video, 
music, etc., col. 11, line 66 through col. 12, line 3 et seq.). 

It would have been obvious to one of ordinary sill in the art at the time of the 
invention to provide personalization of a user's multimedia viewing based upon their 
past recorded behavior, since this would allow a system to render multimedia to a user 
in a manner consistent with that user's preferences, without the need for the user to 
manually customize the rendering each time a user accesses multimedia. 



20. Regarding claim 12, Bahn additionally teaches a computer-readable medium 
containing a computer program product substantially as claimed, enabling a 
programmable device, when executing said computer program product, to: 

a) identify a user's usage of a first content of a first type (see disclosure that the 

system allows a user to customize audio content on interactive television, 
col. 2, lines 8-10 et seq.); 

b) identify that the user concurrently uses a second content of a second type 

when using said first content of the first type (see disclosure that the system 
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allows a user to customize audio content on interactive television, col. 2, 
lines 8-10 et seq.), said second content being unrelated with the first content 
(see disclosure that, for example, the user can elect to listen to jazz style 
music while viewing content on a shopping channel accessed over 
interactive television, col. 2, lines 33-35); and 
c) associate said second content with the first content (see disclosure that once 
the user's selections have been made, the selections can be stored as user 
preferences in an audio library collection and then applied during 
subsequent viewing of the shopping channel, col. 6, lines 40-43 et seq.). 

Bahn does not explicitly teach a computer-readable medium containing a 
computer program product for operating with different types of media content wherein 
the association between said first content and said second content is made on the basis 
of historical usage of the user. 

Errico et al., however, teaches the monitoring and recording of a user's viewing 
and listening habits, and the rendering of desirable audio, image and/or video 
information in accordance with said user's historical viewing and listening habits (see 
col. 4, lines 25-63; see also col. 11, lines 34-45; see also col. 12, lines 25-32; see also 



Application/Control Number: 10/528,681 Page 20 

Art Unit: 2167 

disclosure that the management of individual programs and other data includes 
relationships between any one of the user, the audio, and the images in relation to one 
or more of a program description scheme and a user description scheme, col. 7, lines 6- 
9; see also disclosure that appliances that can be personalized may access content from 
different sources, or access multiple or different types of single media such as video, 
music, etc., col. 11, line 66 through col. 12, line 3 et seq.). 

It would have been obvious to one of ordinary sill in the art at the time of the 
invention to provide personalization of a user's multimedia viewing based upon their 
past recorded behavior, since this would allow a system to render multimedia to a user 
in a manner consistent with that user's preferences, without the need for the user to 
manually customize the rendering each time a user accesses multimedia. 

21. Regarding claim 3, Bahn additionally teaches a system further comprising 
selection means arranged to select the content (see disclosure that the user may choose 
the audio content to be rendered, col. 2, lines 31-33 et seq.). 
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22. Regarding claim 4, Bahn additionally teaches a system wherein said selection 
means are further arranged to identify the first content upon selection of the associated 
second content and/or to identify the second content upon selection of the associated 
first content (see disclosure that after changing the default audio settings while viewing 
a shopping channel, the user's selections can be stored as user preferences in an audio 
library collection and then applied during subsequent viewing of the shopping channel, 
col. 6, lines 19-51 et seq.). 

23. Regarding claim 6, Bahn additionally teaches a system further comprising 
output means arranged to simultaneously output said associated first and second 
content (see disclosure that the system allows the simultaneous rendering of jazz style 
music while the user views content on a shopping channel, col. 2, lines 33-36). 

24. Regarding claim 7, Bahn additionally teaches a system wherein said selection 
means are further arranged to user-operably modify said meta-data (see disclosure that 
the user may upload audio and/or video content, col. 7, lines 15-51; see also disclosure 
that the system performs explicit profiling, col. 7, lines 52-67). 
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25. Regarding claim 8, Bahn additionally teaches a system wherein said identifying 
means is arranged to identify a user's usage of a third content of a second or other type, 
said usage being concurrent to said user's usage of the first content, and said third 
content being unrelated with the first content, and wherein said associating means is 
arranged to associate said third and first content (see disclosure of the association of 
additional content or a variety of different types, col. 3, lines 3-14), the system further 
comprising rating means arranged to rate said association of the first content with the 
second content and/or with the third content (see disclosure of both explicit and implicit 
profiling of the user, col. 7, line 52 through col. 8, line 15). 

26. Regarding claim 9, Bahn additionally teaches a system comprising a plurality of 
devices, each device including output means arranged to output at least one type of the 
media content, and/or input means arranged to obtain at least one type of the media 
content (see disclosure of a variety of devices, col. 5, lines 4-31 et seq.). 

27. Regarding claim 10, Bahn additionally teaches a system wherein said first and 
second content correspond to video and audio content (see disclosure that the system 
allows a user to customize audio content on interactive television, col. 2, lines 8-10 et 
seq.). 
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28. Claims 1 and 3-12 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Ichino (U.S. Patent 5,440,351) in view of Errico et al. (U.S. Patent 7,055,168). 

29. Regarding claim 1, Ichino teaches a system for operating with different types of 
media content as claimed, the system being arranged to enable a user to use a first 
content of a first type (see disclosure that the system allows a user to render audio 
content from a specific radio station concurrently with television content, col. 7, lines 
49-53 et seq.), characterized in that the system comprises: 

a) identifying means for identifying that the user concurrently uses a second 

content of a second type when using said first content of the first type (see 
disclosure that the system allows a user to select the radio frequency to be 
rendered with television content, col. 8, lines 20-24 et seq.), said second 
content being unrelated with the first content (see disclosure that the 
system allows a user to select any radio frequency to be rendered with 
television content, col. 3, lines 31-35 et seq.); and 



Application/Control Number: 10/528,681 Page 24 

Art Unit: 2167 

b) associating means for associating said second content with the first content 
(see disclosure that the system includes programmable memory with the 
capability to store a single television channel AM radio frequency 
association or a number of such associations, col. 8, lines 2-5 et seq.). 

Bahn does not explicitly teach a system for operating with different types of 
media content wherein the association between said first content and said second 
content is made on the basis of historical usage of the user. 

Errico et al v however, teaches the monitoring and recording of a user's viewing 
and listening habits, and the rendering of desirable audio, image and/or video 
information in accordance with said user's historical viewing and listening habits (see 
col. 4, lines 25-63; see also col. 11, lines 34-45; see also col. 12, lines 25-32; see also 
disclosure that the management of individual programs and other data includes 
relationships between any one of the user, the audio, and the images in relation to one 
or more of a program description scheme and a user description scheme, col. 7, lines 6- 
9; see also disclosure that appliances that can be personalized may access content from 
different sources, or access multiple or different types of single media such as video, 
music, etc., col. 11, line 66 through col. 12, line 3 et seq.). 
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It would have been obvious to one of ordinary sill in the art at the time of the 
invention to provide personalization of a user's multimedia viewing based upon their 
past recorded behavior, since this would allow a system to render multimedia to a user 
in a manner consistent with that user's preferences, without the need for the user to 
manually customize the rendering each time a user accesses multimedia. 

Ichino furthermore does not explicitly teach a system further comprising storage 
means arranged to store meta-data comprising information pertaining to said 
associated first and second content. 

Errico et al., however, teaches the storage of meta-data comprising information 
pertaining to said associated first and second content (see disclosure of the storage of 
the Program Description Scheme, beginning at col. 15, line 28; see also disclosure of the 
storage of User Description Scheme, beginning at col. 22, line 27). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to store meta-data comprising information pertaining to said associated first 
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and second content, since without storing such meta-data, it would not be possible for 
the system to render multimedia to a user consistent with said user's preferences. 

30. Regarding claim 11, Ichino teaches a method of operating with different types of 
media content, the method comprising the step of identifying a user's usage of a first 
content of a first type (see disclosure that the system allows a user to render audio 
content from a specific radio station concurrently with television content, col. 7, lines 
49-53 et seq.), characterized in that the method further comprises a step of identifying 
that the user concurrently uses a second content of a second type when using said first 
content of the first type (see disclosure that the system allows a user to select the radio 
frequency to be rendered with television content, col. 8, lines 20-24 et seq.), said second 
content being unrelated with the first content (see disclosure that the system allows a 
user to select any radio frequency to be rendered with television content, col. 3, lines 31- 
35 et seq.), and a step of associating said second content with the first content (see 
disclosure that the system includes programmable memory with the capability to store 
a single television channel AM radio frequency association or a number of such 
associations, col. 8, lines 2-5 et seq.). 
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Bahn does not explicitly teach a method of operating with different types of 
media content wherein the association between said first content and said second 
content is made on the basis of historical usage of the user. 

Errico et al., however, teaches the monitoring and recording of a user's viewing 
and listening habits, and the rendering of desirable audio, image and/or video 
information in accordance with said user's historical viewing and listening habits (see 
col. 4, lines 25-63; see also col. 11, lines 34-45; see also col. 12, lines 25-32; see also 
disclosure that the management of individual programs and other data includes 
relationships between any one of the user, the audio, and the images in relation to one 
or more of a program description scheme and a user description scheme, col. 7, lines 6- 
9; see also disclosure that appliances that can be personalized may access content from 
different sources, or access multiple or different types of single media such as video, 
music, etc., col. 11, line 66 through col. 12, line 3 et seq.). 

It would have been obvious to one of ordinary sill in the art at the time of the 
invention to provide personalization of a user's multimedia viewing based upon their 
past recorded behavior, since this would allow a system to render multimedia to a user 
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in a manner consistent with that user's preferences, without the need for the user to 
manually customize the rendering each time a user accesses multimedia. 



31. Regarding claim 12, Ichino additionally teaches a computer-readable medium 
containing a computer program product substantially as claimed, enabling a 
programmable device, when executing said computer program product, to: 

a) identify a user's usage of a first content of a first type (see disclosure that the 

system allows a user to render audio content from a specific radio station 
concurrently with television content, col. 7, lines 49-53 et seq.); 

b) identify that the user concurrently uses a second content of a second type 

when using said first content of the first type (see disclosure that the system 
allows a user to select the radio frequency to be rendered with television 
content, col. 8, lines 20-24 et seq.), said second content being unrelated with 
the first content (see disclosure that the system allows a user to select any 
radio frequency to be rendered with television content, col. 3, lines 31-35 et 
seq.); and 

c) associate said second content with the first content (see disclosure that the 

system includes programmable memory with the capability to store a 
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single television channel AM radio frequency association or a number of 
such associations, col. 8, lines 2-5 et seq.). 

Bahn does not explicitly teach a computer-readable medium containing a 
computer program product for operating with different types of media content wherein 
the association between said first content and said second content is made on the basis 
of historical usage of the user. 

Errico et al., however, teaches the monitoring and recording of a user's viewing 
and listening habits, and the rendering of desirable audio, image and/or video 
information in accordance with said user's historical viewing and listening habits (see 
col. 4, lines 25-63; see also col. 11, lines 34-45; see also col. 12, lines 25-32; see also 
disclosure that the management of individual programs and other data includes 
relationships between any one of the user, the audio, and the images in relation to one 
or more of a program description scheme and a user description scheme, col. 7, lines 6- 
9; see also disclosure that appliances that can be personalized may access content from 
different sources, or access multiple or different types of single media such as video, 
music, etc., col. 11, line 66 through col. 12, line 3 et seq.). 
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It would have been obvious to one of ordinary sill in the art at the time of the 
invention to provide personalization of a user's multimedia viewing based upon their 
past recorded behavior, since this would allow a system to render multimedia to a user 
in a manner consistent with that user's preferences, without the need for the user to 
manually customize the rendering each time a user accesses multimedia. 

32. Regarding claim 3, Ichino additionally teaches a system further comprising 
selection means arranged to select the content (see disclosure that the user may select 
the frequency of the AM tuner, col. 8, lines 20-24 et seq.). 

33. Regarding claim 4, Ichino additionally teaches a system wherein said selection 
means are further arranged to identify the first content upon selection of the associated 
second content and/or to identify the second content upon selection of the associated 
first content (see disclosure that when the TV/Radio button is pressed, the system 
searches memory to see if it contains an entry for a television channel corresponding to 
the currently-active channel, and if so, sets the AM radio tuner to the associated radio 
frequency, col. 9, lines 7-20). 
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34. Regarding claim 5, Ichino additionally teaches a system wherein said selection 
means are further arranged to function as a recommender for recommending the 
associated first or second content upon a user-operable selection of one of said 
associated second and first content, respectively, using said selection means (see 
disclosure that duplicate associations between radio frequencies and television channels 
may be stored, and the viewer would be allowed to scroll through the associations in 
order to select the desired frequency, col. 10, lines 17-25). 

35. Regarding claim 6, Ichino additionally teaches a system further comprising 
output means arranged to simultaneously output said associated first and second 
content (see disclosure that the selected AM radio broadcast is reproduced through the 
television sound system, col. 9, lines 14-20). 

36. Regarding claim 7, Errico et al. additionally teaches a system wherein the meta- 
data is user-modifiable (see disclosure that the User Description Scheme, which stores a 
user's viewing preferences, can be updated manually by the user, col. 6, lines 42-44 et 
seq.). 
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37. Regarding claim 8, Ichino additionally teaches a system wherein said identifying 
means is arranged to identify a user's usage of a third content of a second or other type, 
said usage being concurrent to said user's usage of the first content, and said third 
content being unrelated with the first content, and wherein said associating means is 
arranged to associate said third and first content (see disclosure of the association of 
duplicate television channel/radio frequency associations, col. 10, lines 17-25), the 
system further comprising rating means arranged to rate said association of the first 
content with the second content and/or with the third content (see disclosure of the 
explicit selection of secondary audio content by radio frequency, thus rating the 
association positively, col. 8, lines 20-24 et seq.). 

38. Regarding claim 9, Ichino additionally teaches a system comprising a plurality of 
devices, each device including output means arranged to output at least one type of the 
media content, and/or input means arranged to obtain at least one type of the media 
content (see disclosure of both a television and a radio tuner, col. 5, lines 40-66 et seq.). 

39. Regarding claim 10, Ichino additionally teaches a system wherein said first and 
second content correspond to video and audio content (see disclosure that the system 
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allows a user to render audio content from a specific radio station concurrently with 
television content, col. 7, lines 49-53 et seq.). 



40. Claims 4 and 6 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Billmaier (U.S. Patent 6,630,963) in view of Errico et al. (U.S. Patent 7,055,168) as 
applied to claims 1, 3 and 7-12 above, and further in view of Bahn (U.S. Patent 
7,162,728). 

41. Regarding claim 4, Billmaier and Errico et al. teach a system for operating with 
different types of media content substantially as claimed. 

Neither Billmaier nor Errico et al. explicitly teaches a system wherein said 
selection means are further arranged to identify the first content upon selection of the 
associated second content and/or to identify the second content upon selection of the 
associated first content. 

Bahn, however, teaches a system wherein said selection means are further 
arranged to identify the first content upon selection of the associated second content 
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and/or to identify the second content upon selection of the associated first content (see 
disclosure that after changing the default audio settings while viewing a shopping 
channel, the user's selections can be stored as user preferences in an audio library 
collection and then applied during subsequent viewing of the shopping channel, col. 6, 
lines 19-51 et seq.). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to register a user's preferences for which media to be played concurrently 
with another media, and to apply said preferences to subsequent content rendering, 
since this would save the user the need to repeatedly select their preferred media every 
time content is being rendered. 

42. Regarding claim 6, Billmaier additionally teaches a system further comprising 
output means arranged to simultaneously output said associated first and second 
content (see disclosure that the system allows the simultaneous rendering of different 
content received from different sources, col. 2, lines 44-57; see also col. 6, line 63 through 
col. 7, line 3). 
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43. Claim 5 is rejected under 35 U.S.C. 103(a) as being unpatentable over Billmaier 
(U.S. Patent 6,630,963) in view of Errico et al. (U.S. Patent 7,055,168) in view of Bahn 
(U.S. Patent 7,162,728) as applied to claims 4 and 6 above, and further in view of Kotz et 
al. (U.S. Patent Application Publication 20040068552). 



44. Regarding claim 5, Billmaier, Errico et al. and Bahn teach a system for operating 
with different types of media content substantially as claimed. 

None of Billmaier, Errico et al. nor Bahn explicitly teaches a system wherein 
said selection means are further arranged to function as a recommender for 
recommending the associated first or second content upon a user-operable selection of 
one of said associated second and first content, respectively, using said selection means. 

Kotz et al., however, teaches a system wherein said selection means are further 
arranged to function as a recommender for recommending the associated first or second 
content upon a user-operable selection of one of said associated second and first 
content, respectively, using said selection means (see disclosure that the system 
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provides recommendations to a user based upon the user's preferences, past selections 
and user location, paragraph [0010] et seq.). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to provide the user the ability to choose from a list of recommended content, 
since this would give the user the ability to choose from among a variety of content that 
has been judged to be of potential interest to the user based on their profile and past 
selections, thus providing access to content that the user might not have chosen 
explicitly, perhaps because they were not aware of the content, but would be of interest 
to the user. 



45. Claim 5 is rejected under 35 U.S.C. 103(a) as being unpatentable over Bahn (U.S. 
Patent 7,162,728) in view of Errico et al. (U.S. Patent 7,055,168) as applied to claims 1, 3, 
4 and 6-12 above, and further in view of Kotz et al. (U.S. Patent Application Publication 
20040068552). 
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46. Regarding claim 5, Bahn and Errico et al. teach a system for operating with 
different types of media content substantially as claimed. 

Neither Bahn nor Errico et al. explicitly teaches a system wherein said selection 
means are further arranged to function as a recommender for recommending the 
associated first or second content upon a user-operable selection of one of said 
associated second and first content, respectively, using said selection means. 

Kotz et al., however, teaches a system wherein said selection means are further 
arranged to function as a recommender for recommending the associated first or second 
content upon a user-operable selection of one of said associated second and first 
content, respectively, using said selection means (see disclosure that the system 
provides recommendations to a user based upon the user's preferences, past selections 
and user location, paragraph [0010] et seq.). 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to provide the user the ability to choose from a list of recommended content, 
since this would give the user the ability to choose from among a variety of content that 
has been judged to be of potential interest to the user based on their profile and past 
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selections, thus providing access to content that the user might not have chosen 
explicitly, perhaps because they were not aware of the content, but would be of interest 
to the user. 

Response to Arguments 

47. Applicant's arguments with respect to claims 1 and 3-12 have been considered, 
but are not persuasive. 

48. The Applicants have argued that the Errico et al. reference fails to teach storing a 
user's viewing and listening habits with regard to the concurrent use of two different 
types of content. 

49. In response to applicant's arguments against the references individually, one 
cannot show nonobviousness by attacking references individually where the rejections 
are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 
(CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). 

50. In this case, each of the primary references teaches the identification that a user 
concurrently uses a second content of a second type when using a first content of a first 
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type, said second content being unrelated to said first content (see Billmaier's 
disclosure that a user selects a secondary audio program to replace the primary audio 
program associated with a television transmission, col. 9, lines 1-10 et seq.; see also 
disclosure that the secondary audio content is transmitted via the Internet, col. 2, lines 
44-51 et seq.; the examiner notes that Applicants' specification discloses that the term 
'unrelated' should be interpreted as meaning that the system has no data indicating the 
relation between the contents, page 3, lines 21-24; although the television and secondary 
audio content might both be broadcasting the same event, they are independently 
produced and broadcast, and there is no reference in either data stream to the other, 
which means that for the purposes of the claimed invention, the video and audio 
broadcasts are unrelated; the fact that the buffering period of the video stream must be 
manually calibrated in order to match the audio stream provides further evidence that 
the streams are unrelated; see also Bahn's disclosure that the system allows a user to 
customize audio content on interactive television, col. 2, lines 8-10 et seq.; see also 
disclosure that, for example, the user can elect to listen to jazz style music while 
viewing content on a shopping channel accessed over interactive television, col. 2, lines 
33-35; see also Ichino's disclosure that the system allows a user to select the radio 
frequency to be rendered with television content, col. 8, lines 20-24 et seq.; see also 
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disclosure that the system allows a user to select any radio frequency to be rendered 
with television content, col. 3, lines 31-35 et seq.). 

The Errico et al. reference is relied upon only to teach the recording and use of 
information about a user's preferences regarding their viewing and listening habits. 

When combined, the references teach a system that recognizes that a user uses 
two different media of different media types concurrently, records and maintains 
historic information regarding said uses, and uses that information to automatically 
render information to the user consistent with the user's viewing and listening 
preferences. 
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Conclusion 

51. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of 
time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and 
any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date 
of the advisory action. In no event, however, will the statutory period for reply expire 
later than SIX MONTHS from the mailing date of this final action. 
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Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Luke S. Wassum whose telephone number is 571-272- 
4119. The examiner can normally be reached on Monday-Friday 8:30-5:30, alternate 
Fridays off. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John R. Cottingham can be reached on 571-272-7079. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

In addition, INFORMAL or DRAFT communications may be faxed directly to the 
examiner at 571-273-4119, or sent via email at hike, w assu m^u sp to. gov, with a previous 
written authorization in accordance with the provisions of MPEP § 502.03. Such 
communications must be clearly marked as INFORMAL, DRAFT or UNOFFICIAL. 

Customer Service for Tech Center 2100 can be reached during regular business 
hours at (571) 272-2100, or fax (571) 273-2100. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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Primary Examiner 
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